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Information theory provides shortcuts which allow to deal with complex systems. The basic 
idea one uses for this purpose is the maximum entropy principle developed by Jaynes. However, an 
extensions of this maximum entropy principle to systems far from thermal equilibrium or even to non- 
physical systems is problematic because it requires an adequate choice of constraints. In this paper 
we apply the information theory in an even more abstract way and propose an information transfer 
model of natural processes which requires no choice of adequate constraints. It is, therefore, directly 
apphcable to systems far from thermal equilibrium and to non-physical systems. We demonstrate 
that the information transfer model yields well known laws, which, as yet, have not been directly 
related to information theory, such as the radioactive decay law, Fick's first law and Hubble's law. 



I. INTRODUCTION 

It is generally accepted that, from a unifying point of 
view, information theory provides shortcuts which allow 
to deal with complex systems [l|. The basic idea one 
uses for this purpose is the maximum entropy principle 
developed by Jaynes 2] . We repeat here the summary of 
the basic ideas as formulated by Haken ([H p. 33): We 
start from macro-observables which may fluctuate and 
whose mean values are known. We distinguish the macro- 
variables by an index k and denote their mean values by 
/fc. We wish then to make a guess at the probability 
distribution pj of the system over states labelled by the 
index j. This is achieved under the maximization of the 
information i 

i = - Y^Pjlnpj (1) 

j 

under the constraint that 

E^'^'/f-^ (2) 

j 

Evidently, /j'''' is the contribution of state j to the macro- 
variable labelled by k. Furthermore we require 



(3) 



i.e. that the probability distribution pj is normalized 
to unity. As was shown by Jaynes this principle al- 
lows us to derive the basic formulae of thermodynamics 
in a very short and elegant way. For this derivation the 
constraints refer to the conserved quantities of a closed 
system, i.e. energy, particle numbers etc. The crux of 
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the problem of extending this maximum entropy princi- 
ple to systems far from thermal equilibrium or even to 
non-physical systems lies in the adequate choice of con- 
straints... (see [H p. 33 for further discussion). 

This summary shows that an extension of the maxi- 
mum entropy principle to systems far from thermal equi- 
librium or to non-physical systems is problematic because 
it requires the choice of adequate constraints. In this pa- 
per we will apply the information theory in an even more 
abstract way which requires no choice of adequate con- 
straints. For that it is convenient to apply the definition 
of information which was originally given by Hartley H. 
He described the information / by the number of selec- 
tions, n, 



I = K, 



(4) 



where = Kq In s is a constant which depends on the 
number of symbols, s, available at each selection and Kq 
defines the unit of information. If Kq = 1/ In 2 the unit of 
information is given in bits. Taking into account the defi- 
nition i = I /{nK{}) and setting = 1/s = constant with 
j = 1 ... s we get equation ([4]) also from equation ([T]). 
That is, Hartley's equation (jH) describes the information 
if the probability to use a symbol is equal for all symbols. 



II. INFORMATION TRANSFER MODEL 

It is well known from information theory Q that any 
technical process can transfer at best the complete infor- 
mation Iq from an information source q to an information 
destination u. That is, we have for the information /„, 
which the information destination u receives from the in- 
formation source g, the inequality 



Iu<In 



(5) 
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Fig. [T] shows a simplified version of Shannon's diagram 
of a general communication system (see Fig. 1 in Shan- 
non's work [3|) where the transfer system comprises a 
transmitter, a channel, a receiver and a noise source. We 
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FIG. 1: Let the information source q have the information Iq. 
Any transfer system can at best transfer the information Iq 
to the information destination u, so that we have generally 
the inequality /„ < Iq. 



are familiar with information transfer during a technical 
process: The information source q could be for example a 
computer, labelled q, which communicates via a transfer 
system (e.g. via the world wide web) with a computer, 
labelled u. One would say in this case that there is in- 
formation transfer from the computer q to the computer 
u. 

The simplified diagram in Fig. [T] allows us to include 
formally also natural processes in the transfer system. 
To distinguish this approach clearly from mere techni- 
cal communication systems we will call it more generally 
information transfer model (of natural processes). We 
make no detailed assumptions concerning the nature of 
the process but will restrict the considered processes by 
condition [TJ 

Condition 1 The considered natural process can be suf- 
ficiently described by only two independent generic pro- 
cess variables {q, u) and is able to transfer information. 

The values of the generic process variables are in the 
general case not restricted so that we have the condition 



Condition 2 —oo<q< +oo and — cx) < u < +oo 

Because the two variables are independent we can for- 
mally assign them to the information source q and the in- 
formation destination u of the information transfer model 
(Fig. [1]). That is, in the context of the proposed infor- 
mation transfer model we now introduce a new point of 
view: We say that there is information transfer from the 
variable q to the variable u. It is, therefore, convenient 
to call variable q the source variable and variable u the 
destination variable. 

We must now find a definition of information which 
is directly related to the generic variables of the con- 
sidered process. Take for example a rod of length \Aq\ 
from which we cut (select) n — \Aq\ / \Sq\ pieces (num- 
ber of length selections, n) of smaller length \6q\. It is 
obvious that we have the condition n > 1. It is also a 
well known fact that the length \Sq\ will become quan- 
tized for microscopic values of \dq\ so that we have the 
further condition \dq\ > 0. That is, we cannot select 
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FIG. 2: Graphical illustration of symbols and naming con- 
ventions to define process variable selections = |Ag| / \Sq\ 
of a natural process. The generic process variable q can have 
values from — oo to -|-oo. The considered natural process is 
quantitatively described by an absolute value | Aq| of the pro- 
cess variable. The symbol \5q\ represents the absolute value of 
the process signal (see text for an explanation of its meaning). 



an infinite number of pieces, n, from a rod of limited 
length (|Aq| < oo). The only possible way to select an 
infinite number of pieces (n —>■ oo) is given if we select 
pieces of limited length {\6q\ < oo) from a rod of infinite 
length (|Aq| oo). In this special example we consid- 
ered length selections. Generalizing we now define pro- 
cess variable selections 



\Sq\ 



> 1 and 



> 1 



(6) 



where Uq is the number of source variable selections and 
n„ is the number of destination variable selections. \Aq\ 
and I Am I are absolute values of the process variables and 
\Sq\ and \Su\ are absolute values of the process signals. A 
graphical illustration of symbols and naming conventions 
to define process variable selections is given by Fig. [2l 

The notation "process signal" is motivated by the cir- 
cumstance that we need a detector to detect these abso- 
lute values so that we define the term 



\\5u\J 



detector 



\\Sq\ 



(7) 



detector 



and call it detector. As an example consider the detector 
p — {\SW\ 1 1 I) detector whcrc the detector value is the 
pressure p of gas particles in a box of volume V . That 
is, this detector relates an absolute value of the process 
source signal, \SiW\, "emitted" from the process work W 
to an absolute value of the process destination signal, 
\SV\^ "detected" at the process volume V ^ and delivers 
as output value the pressure p. From this example it 
becomes obvious that a pressure of p = is contradictory 
to the considered process, i.e. gas particles in a box at 
a given temperature, because a pressure of p = means 
that there is not any gas particle in the box. To avoid this 
contradiction we have to exclude \SW\ — and \ W\ — oo 
in this special example. Generalising we get the following 
condition for the absolute values of the process signals 

Condition 3 < \Sq\ < oo and < \5u\ < oo 
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According to Hartley's definition ([4]) of information we 
can now define an information / which is directly related 
to the two independent process variables q and u 



\5u\ 



(8) 



Applying inequality ([5]) we have the general inequality 



\5u\ 



(9) 



where the constants Kj and i^" are related to the given 
natural process. Because we make no detailed assump- 
tions about the nature of the process we must consider 
Kj and as unknown constants in our model. For 
that we define the dimensionless positive process depen- 
dent constant k 



< K 



< oo 



(10) 



and call it the information transfer index of a process. 
Using the constant k, inequality ^ becomes 



K n„ < rio or k 



\Au\ 



< 



|Ag| 



\6u\ - \Sq\ 



(11) 



A. The ideal information transfer process 

In the case of ideal information transfer the complete 
information Iq from the information source q is sent to 
the information destination u so that we have the condi- 
tion 

Condition 4 /„ = Iq 

We refer to this condition as ideal information transfer 
condition. It is now useful to define the corresponding 
information transfer process. 

Definition 1 An Ideal Information Transfer (IIT) pro- 
cess is a natural process which fulfils condition [7] and the 
ideal information transfer condition^ That is, we have 
a natural process which enables ideal information trans- 
fer (lu — Iq) from the source variable q to the destination 
variable u. 

Because it is forbidden that any process (technical or 
natural) transfers more information to the destination 
than the source sends (that is, > Iq or > Uq is 
forbidden) one can formulate a sufficient condition for a 
global IIT process. 

Condition 5 // any natural process can be described 
globally by two independent absolute values of the global 
process variables Qgiobai — |Ag| and Wgiobai = l^^^l; it 
must become an IIT process according to definition [7] for 



a sufficiently large absolute value of the global destination 
variable (ugiobai ~^ oo) if the absolute value of the global 
source variable is limited (ggiobai < oo)- If o, natural pro- 
cess is a global IIT process for a given large absolute value 
of the global destination variable, Mgiobai ~ l^"*^! j *^ stays 
a global IIT process for Ugiobai > u*^ioh!>\- 

We will call this condition the global IIT process condi- 
tion. In an analogous way one can formulate a sufficient 
condition for a local IIT process. 

Condition 6 We consider any natural process which is 
described locally by two independent absolute values of 
the process variables (jAgj , |Am|). If there is locally a suf- 
ficiently large number of destination variable selections 
n.u given so that we have nn^ » 1, the natural pro- 
cess must become an IIT process according to definition 
[7] if the absolute value of the source variable becomes lo- 
cally sufficiently small {\Aq\ — > 0). If a natural process 
is a local IIT process for a given small absolute value of 
the source variable, |Ag|*, it stays a local IIT process for 
\Aq\ < \Aq\*. 

We will call this condition the local IIT process condition. 
The justification of conditions [5] and [6] is given in the 
appendix. 

For an IIT process according to definition [1] we get 
from inequality (jlip the following equation 



M 

1*^^! / detector 



1 |Ag| 
K \Au\ 



with < K < oo 



(12) 



where \Aq\ is the absolute value of the source variable, 
Am I is the absolute value of the destination variable and 
K the information transfer index. It is useful to call this 
equation the ideal detector equation (of an IIT process). 
The notation "detector" is a synonym for any (real or 
virtual) measurement device which delivers the output 
value {\Sq\ / \Su\)^^^^^^^^j. in an experiment. Because of 
equation (fT^ we have to measure the absolute value of 
only one process variable and can calculate the absolute 
value of the conjugate process variable by using the out- 
put value of the detector device (if the natural process is 
an IIT process and if we know the information transfer 
index n). 



B. Ideal transfer laws of the IIT process 

We now ask which absolute value of the destination 
variable we can expect for a given absolute value of the 
source variable if we consider an IIT process. That is we 
are looking for a functional relation like 



{\Au\)^Fi\Aq\) 



(13) 



where (|Am|) is the expected absolute value of the des- 
tination variable and |Ag| is a given absolute value of 
the information source variable. We will call such a 
functional relation between the generic process variables 
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an ideal transfer law if it results from ideal information 
transfer and is hence related to an IIT process. To de- 
rive the function F in equation ([T3|) we postulate that 
the absolute values of the process signals {\Sq\ , \Su\) be 
small enough to be considered as infinitesimal increments 
{\d {Aq)\ , \d (Au)|) of the absolute values of the process 
variables (I I , |Am|). This means that the process vari- 
able selections must be sufficiently large 



Condition 7 



^9 = wf » ^ and n„ = ^ » 1 



We will refer to this condition as the ideal transfer law 
condition. The ideal detector equation (|12p now becomes 



M 

\Su\ 



detector 



M(Ag)| 
\d{Au)\ 



\dq[ 
\du\ 



l\Aq[ 
K \Au\ 



(14) 



If we integrate equation we get the ideal transfer law 
we are looking for. Before we do this, it is important to 
distinguish strictly between so called constant and float- 
ing information sources so that we define: 

Definition 2 A constant information source is given if 
the absolute value of the information source variable 
is constant, \Aq\ — |Ago|; 'in the ideal detector equa- 
tion il2\) . If this is not the case a floating information 
source is given. 

For an IIT process with a constant information source 
according to definition [2] equation p4|) becomes 



d\Aq\ l\Aqo\ . 

-7TT— I = ±--nrT '^ith Ago = 

d \ Au\ K \ Au\ 

If we integrate equation (|15p we have 

|Ag| (|Au|) 



constant 



(15) 



± 



I Ago I 



d|Ag|' 



1 



I Am 



■dlAul 



(16) 



|A?„f 



Aujcf 



where (|Am|) is the expected absolute value of the infor- 
mation destination variable and \Aq\ is a given absolute 
value of the information source variable. The subscript 
ref indicates absolute reference values of the IIT pro- 
cess which are unknown in the context of the information 
transfer model. The general solution of equation (|16p is 



(|Au|) = lAurofl exp ±K 



\Aq\ - |Ag, 



rof 



I Ago I 



(17) 



We will refer to this equation as the ideal transfer law of 
a constant information source. One can rearrange equa- 
tion (fT7|) also as follows 



(|Aw|) = I Auol exp y±K 
with |Auo| = |AMi-ot|exp =pK 



|Ag| 
I Ago I 

Aqref 



I Ago I 



(18) 



where |Auo| corresponds to |Ag| = 0. 



For an IIT process with a floating information source 
according to definition [5] equation becomes 



d\Aq\ 
d\Au\ 



= ± 



1 |Ag| 



If we integrate this equation we have 

|A9| (|A«|) 



(19) 



(20) 



|A?„f| 



|A«„ 



where (|Au|) is the expected absolute value of the infor- 
mation destination variable and |Ag| is a given absolute 
value of the information source variable. Solving the in- 
tegrals gives the general solution 



{\Au\) 



±1/k 



-Rn |Ag| 



with i?„ 



K lAUr 



|±1/k 



|Agrof I 



> 



(21) 



We will refer to this equation as the ideal transfer law 
of a floating information source. It is convenient to 
call the reference constant (of an IIT process) be- 
cause it is directly related to the absolute reference val- 
ues (|Agre/| , lAitre/l) which are unknown in the context 
of the information transfer model. 



C. Ideal point particles 

The defining feature of a point particle is being zero- 
dimensional. It is an appropriate representation of any 
physical object whose size, shape, and structure is irrele- 
vant in the given context. Consider for example ideal gas 
particles in a box of volume V. Because ideal gas par- 
ticles have no volume the expected volume, (V), which 
is related to the ideal gas particles is equal to the box 
volume, (V) = V. Generalising we now say that, in the 
context of the information transfer model, ideal point 
particles obey the condition 

Condition 8 (|Au|) = |Au| 

so that we will refer to it as ideal point particle condition. 
Because of condition[5]one can combine the ideal detector 
equation and the ideal transfer law (equation (fTT)) or 
equation (|2ip ') if one considers ideal point particles and 
if the ideal transfer law condition [7] is fulfilled. 

Furthermore, we say that in the context of the infor- 
mation transfer model ideal point particles enable an 
ideal information transfer and are hence related to an 
IIT process according to definition[T] The available num- 
ber of process variable selections ^ , which are the cen- 
tral quantities to express the transferred information, can 
then directly be related to the available number of ideal 
point particles, iVideai, so that we have in the case of ideal 
information transfer, /„ — Iq, the relation 



|A^| _ |Ag| 
\5u\ K\5q\ 



ideal 



(22) 
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We will refer to this equation as ideal point particle equa- 
tion and we will in the following discussion always con- 
sider a huge number of ideal point particles, Mdeai >> 1, 
so that the ideal transfer law condition [7| is always ful- 
filled. 



III. GAS PARTICLES IN A BOX 

We now demonstrate that one can derive the ideal gas 
law if one applies the global IIT process condition [51 For 
that we consider N real gas particles in a box. An impor- 
tant quantity to describe the N real gas particles is the 
gas pressure, p, which is measured by a pressure detector 



P ■ 



\5W\ 



(23) 



detector 



where W is the work which contributes to the change of 
the gas box volume V . This detector defines two indepen- 
dent process variables [W, V). To proceed the discussion 
we must assume that our considered process, moving real 
gas particles in a box, fulfils condition[T] Any real gas can 
deliver only a limited absolute value of work so that we 
can consider this work as the global information source 
of the process {W — (jgiobai — |Ag| < oo). Consequently, 
we can consider the volume as the global information 
destination of the process. According to the global IIT 
process condition [S] the process, moving real gas parti- 
cles in a box, must become a global IIT process if the 
absolute value of the global destination variable, V, is 
sufhciently large {V = itgiobai = oo). For an IIT 

process our pressure detector ([^ will deliver an output 
value according to the ideal detector equation (set- 
ting ggiobai = lAg| =W and Ugiobai = \/S.u\ = V) 



\5W\ 



detector 



1 w 



(24) 



where the term W/k can be determined by this equa- 
tion if we measure p and V. Condition [5] is a sufficient 
condition to justify ideal information transfer and, there- 
fore, equation (|24p . A more descriptive explanation for 
the ideal information transfer is that, if the volume V 
becomes sufficiently large, the average spacing between 
the N real gas particles becomes large enough so that 
the real gas particles can be considered as an ensemble 
of point particles with no form of interaction (ideal gas 
particles). Its feature is then to transfer information in 
an ideal way (condition [4]) so that the process, moving 
real gas particles in a box, becomes an IIT process ac- 
cording to definition [1] 

For an IIT process we can also apply the ideal point 
particle equation ([22]) and get in this case 



1 W 



V 



\SV\ k\6W\ 



= TV » 1 



(25) 



where N is the number of ideal gas particles in a box of 
volume V. Considering the ideal detector equation 



of the IIT process we have the relation 
W 

pV^ — ^N\6W\ (26) 

and from thermodynamics we know that for an ideal gas 

pV^NksT (27) 

where fc^ is the Boltzmann constant and T the tempera- 
ture. Comparing equations ([26)1 and ([27)1 we see that the 
unknown absolute value of the process (energy) signal, 
\SW\, is given by 



\SW\ =kBT 



(28) 



and is hence proportional to the temperature T . From 
equation ([26]) we can calculate the average absolute work 
value per gas particle 



W 

ly = — ^k\5W\ =KkBT 



(29) 



and know from thermodynamics that k = 3/2 for an ideal 
monatomic gas. 



A. The isothermal volume change 

We consider an isothermal volume change of the ideal 
gas. That is, our gas box is thermally in good contact to 
the surroundings so that the (initial) energy W remains 
constant [W = Wq = constant) if we start to change the 
initial volume V. According to the global IIT process 
condition [5| the process, moving real gas particles in a 
box, remains a global IIT process if the absolute value of 
the global destination variable, V, stays sufficiently large 
during the isothermal volume change. For an IIT process 
we can apply the ideal transfer law p?| of a constant 
information source (setting |Ago| — Wq, \Au\ = V). 



f Aiy\ 
(l/) = V;cfexp ^n^^ 

with AM^ ^±{W ~ l^ref) 



(30) 



where {V) is the expected value of the gas volume for a 
given work, AVF, and V-cd is the reference volume. The 
ideal transfer law condition [7| is valid because of equa- 
tion ps]) , and the ideal gas particles obey the ideal point 
particle condition [8| so that we have 



{V)^V 

Combining equations ([50)1 and PT|) we get 



AVK 



1 / V 

-W^oln — 

K V Vref 



(31) 



(32) 



The work AW^ is zero if there is no change of the volume. 
However, if the gas (pressure) increases the box volume 
{V > Vrci), we gain mechanical work AW^ > 0. And if 
the box volume decreases {V < V^ef), the gas consumes 
mechanical work AW < Q. 
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B. An adiabatic volume change 

We now consider an adiabatic volume change of the 
ideal gas. That is, our gas box is thermally isolated so 
that the (initial) absolute value of energy, W, begins to 
change (to float" ) if we start to change the initial vol- 
ume V. If we consider only sufficiently large volumes, V, 
the process, moving real gas particles in a box, remains 
an IIT process during our adiabatic volume change ac- 
cording to condition [S] For an IIT process we can apply 
the ideal transfer law of a floating information source 
(using equation ((2T|) and setting Qgiobai = = W and 

Mglobal = |Au| = V). 



{V)-'^^ = -R,W 

K 



(33) 



where the negative sign indicates that W decreases if V 
increases. (V) is the expected gas volume for a given en- 
ergy, W, which contributes to the volume change and 
is the reference constant of the IIT process. The ideal 
transfer law condition [7| is valid because of equation ([^5)1 
and we can again apply the ideal point particle condi- 
tion ((3T|) which allows us to combine equations ((24| and 
([33|) so that we get a relation between the experimentally 
accessible quantities p and V 



pV^ 



1 



constant 



(34) 



This is the well known relation between p and V if an 
ideal gas undergoes a reversible adiabatic change, where 
we know from thermodynamics that 7 = 5/3, and thus 
K = 3/2, for an ideal monatomic gas. Notice that we 
would derive an equivalent relation (|34p with the same 
arguments for polytropic processes. 



IV. CONSTANT INFROMATION SOURCES 

The example of real gas particles in a box has shown 
that real gas particles will "transform" to ideal point par- 
ticles if the gas box volume becomes sufficiently large. It 
is plausible and descriptive to assume that the reason 
for this "transformation" is a sufficiently large distance 
between the real gas particles so that interactions be- 
tween the gas particles become negligible. However, in 
the context of the proposed information transfer model 
such descriptive arguments are not necessary because the 
global IIT process condition [5] is sufficient to explain this 
"transformation" . We see that the argument of ideal in- 
formation transfer is more general than the descriptive 
argument that the interactions between the gas particles 
become negligible. In this subsection we will discuss two 
examples of constant information sources, where we can 
generally expect an exponential law like equation (|17p if 
the information transfer is ideal. Especially the second 
example will show that the descriptive argument of negli- 
gible interactions between point particles is not necessary 



to explain ideal information transfer. In such a case the 
only argument is the global IIT process condition [5] to 
understand why ideal information transfer occurs. 



A. The radioactive decay process of unstable 
atomic nuclei 

We will now discuss the radioactive decay of N unsta- 
ble atomic nuclei where we know that this decay process 
can be described by an exponential radioactive decay law 



(N) = No exp 



|At| 

T 



(35) 



where r is a time constant, {N) is the expect num- 
ber of unstable atomic nuclei at the end of the decay 
time interval, |Ai|, and Nq is the number of unstable 
atomic nuclei at \At\ — 0. The unstable atoms can be 
embedded in a solid medium so that there are interac- 
tions with the stable matrix atoms and/or interactions 
between the unstable atoms. However, it is plausible 
to assume that physical processes in the nuclei of the 
unstable atoms can be considered as independent from 
such interactions. That is, we can idealise the N un- 
stable atomic nuclei as ideal point particles if we con- 
sider their radioactive decay only. An important fea- 
ture of ideal point particles is ideal information trans- 
fer so that we can conclude that equation ([55]) reflects 
an ideal transfer law of a constant information source. 
Comparing equation ([55]) and equation (fT5|) we can ob- 
viously define in the context of the information transfer 
model a constant absolute value of time as the informa- 
tion source, |Ago| = = f^T = constant. Consequently 
the absolute value of the information destination variable 
will be the number N of available unstable atomic nuclei 
{\Au\^N). 

These were descriptive arguments to see why the ra- 
dioactive decay law (|35p reflects an ideal transfer law (fT5)) 
of a constant information source. We will now give a more 
rigorous derivation of equation (|35p by means of the in- 
formation transfer model. Like for the example of the 
real gas particles we start again with a technical detector 
device 



decay rate = 



fm 
\ m 



(36) 



detector 



which measures the decay rate of unstable atomic nuclei 
in this example. Analogously to the pressure detector 
(1^5)1 this detector defines two independent process vari- 
ables {t,N). To proceed the discussion we must assume 
that our considered process, the radioactive decay of un- 
stable atomic nuclei, fulfils condition [TJ According to 
the global IIT process condition [5] the radioactive decay 
process must become a global IIT process if the number 
N of available unstable atoms is sufficiently large. For 
an IIT process we can apply the ideal detector equation 
(fT^ which becomes in this case (setting \Aq\ = to and 
|Am| = N) 
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detector 



kN 

17 



and to — constant 



(37) 



and we can apply the ideal point particle equation ((22 
which becomes in this case 



N 



to 



\dN\ K\St\ 



N >> 1 and to = constant (38) 



We see that the process variable selections, N/ \6N\ or 
to/\St\, are equal or, respectively, proportional to the 
number N of available unstable atomic nuclei. The ab- 
solute values of process signals become, therefore. 



\SN\ = 1 and \St\ 



kN 



(39) 



That is, our detector p6|) will detect one process (decay) 
signal during the absolute value of the process (time) 
signal \5t\ = ^/{kN). The ideal transfer law condition[7| 
is valid because of equation ([55]) so that the ideal transfer 
law (fTSl) becomes in this case 



{N) = TVoexp I ) with r 



which is the decay law ([35|) of unstable atomic nuclei. 
The constant absolute value of the information source, 
to, and the information transfer index n are unknown 
constants for the decay process but are related to the 
measurable time constant r. For the measurement of the 
time constant r we can apply the ideal point particle 
condition El {N) = N, so that we can combine the ideal 
detector equation ([57)1 and the ideal transfer law (|l(I)l 



decay rate 



V m ) 



detector 



^0 / |At| , , , 
_£expf-^) (41) 



This relation between detector and transfer law allows us 
to determine experimentally the time constant r of unsta- 
ble atomic nuclei by a technical realisation of the detector 
and by our laboratory clock which measures |Ai|. 



B. The isothermal vacancy formation process in a 
crystalline solid 

Considering the Gibbs free energy G of a crystal with 
No occupied lattice sites and vacant lattice sites, the 
number of vacancies N!^'^ at equilibrium is given by 



dG 







which yields (e.g. [H p. 27) 



exp 



9f 

ksT 



with 



No + 



(42) 



(43) 



where n'^'^ is the fraction of vacant lattice sites at equi- 
librium, 5/ is the Gibbs free energy for the formation of 
a vacancy, fc^ the Boltzmann constant and T the tem- 
perature. 

In contrast to the radioactive decay process there are 
no descriptive arguments to see that equation ()43|) re- 
flects an ideal transfer law of a constant information 
source. Contrary to statistical thermodynamics the sim- 
ple but universal approach of the information transfer 
model will principally not allow us to understand why 
vacancies come into existence in a crystal. However, the 
information transfer model and especially its global IIT 
process condition [5] will enable us to see that the isother- 
mal vacancy formation process is an IIT process accord- 
ing to definition[T] The most convenient way to apply the 
information transfer model is to start the discussion with 
a technical detector device which corresponds to defini- 
tion ([7]) because such a detector defines immediately two 
independent process variables. If there is no technical de- 
tector available we can define a plausible virtual detector. 
In this case we define 

f\5W\\ 

virtual vacancy detector — — — - (44) 

V 1 /detector 

where W is the work which contributes to the change of 
the number of vacancies, iV, in the crystal. This detector 
is highly plausible because a change of the number of va- 
cancies will require work. To proceed the discussion we 
must assume that the vacancy formation process fulfils 
condition [TJ Like for the ideal gas we consider W as the 
information source variable so that N becomes the in- 
formation destination variable. Our crystal is thermally 
in good contact to the surroundings so that the energy, 
PF, which contributes to the change of the number of va- 
cancies remains always constant [W — Wq — constant). 
That is, we have a constant information source in the 
context of the information transfer model. According to 
the global IIT process condition O the formation pro- 
cess of real vacancies in a crystal must becomes a global 
IIT process if the absolute value of the global destination 
variable, N , becomes sufficiently large. For an IIT pro- 
cess we can apply the ideal transfer law (fT7)) of a constant 
information source (setting jAgol — Wq, |Am| = N) 

(iV) = A..,exp(-.M) 



with AM/ = {W - W,of) 



(45) 



where {N) is the expected number of vacancies for a given 
energy \/SW\ and N^^i is a reference number. \ISW\ is 
simply a "certain amount of energy" in the context of the 
information transfer model. A physical interpretation of 
\/S.W\ requires the application of statistical thermody- 
namics. The setting of the right sign in equation (fT7|) 
requires also some knowledge concerning the considered 
process. In this case we know that the expected number 
of vacancies decreases if |AW^| increases. If the forma- 
tion of vacancies would require no energy, |AVF| = , all 
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lattice sites would be empty, (N) = iVj-ef = Matt- That 
is, as reference number of the considered process we can 
set the available number of lattice sites of the crystals. 

In the case of ideal information transfer we could con- 
sider real gas particles as ideal point particles. In an 
analogous way we can now consider in a crystal the real 
particles located at the lattice sites as ideal point parti- 
cles. If we define a vacancy as a real "particle" located at 
a lattice site it can be considered in the case of ideal in- 
formation transfer, too, as an ideal point particle. That 
is, the sum of all ideal particles is equal to the number 
of all available lattice sites in the crystal, A'^idoai = Matt, 
so that the ideal point particle equation ([22]) becomes 



N 



Wo 



latt 



>> 1 



\6N\ K\dW\ 

and Wo — constant 



(46) 



The ideal transfer law condition [7] is valid because 
of equation (|^ . Considering that we have Wo — 
uNiatt \SW\ and N^ci — Matt, equation (|45)) becomes 

, \^^\\ ■ , 

[n) = CXp I — I With 



(n) 



(N) 



\SW\J 



and I Aw I 



latt 



\AW\ 



latt 



(47) 



where (n) is the expected fraction of vacant lattice sites 
for a given energy jAwj. Comparing equations (j47p and 
(|43p we now can physically assign the energy \Aw\ to 
the Gibbs free energy for the formation of a vacancy, 
\Aw\ = gf. That is, the absolute value of the process 
(energy) signal, for the vacancy formation process 

is then given by 



\SW\ = fcsT 



(48) 



This is a remarkable result because we have seen that 
the absolute value of the process (energy) signal for the 
process of ideal gas particles in a box is also given by 
ksT (see equation Because of condition [3] we have 

< \5W\ < cxD so that in the context of the information 
transfer model temperatures of T = and T — oo are 
not allowed. 



V. THE RANDOM WALK PROCESS 

We now demonstrate that one can derive Pick's first 
law (e.g. [fl p. 1) if one applies the local IIT process 
condition [6] which states that we will have ideal informa- 
tion transfer if the absolute value of the source variable 
is sufficiently small. For that we consider the random 
walk process of point particles. If a certain number N 
of point (tracer) particles are, in a thought experiment, 
suddenly introduced on the yz plane at x = xo, then, 
in the time interval \At\, some particles would progress 



a distance, |Aa;i|; still others, |Aa;2|; etc. The Einstein- 
Smoluchowski equation says that (e.g. p. 380) 



Aa;2 



{Axif + iAx^r 



(Axn 



N 



2D \At\ 



(49) 



where Ax'^ is the mean square distance and D is the dif- 
fusion coefficient. This equation cannot tell us how many 
particles travel a distance, |Aa;i| ; how many, |Aa;2|; etc. 
It is a simple but fundamental law which is related to 
any randomly walking point particles. 

Comparing equation ([49]) with the ideal transfer law 
(PT|) of a floating information source we get a similar 
result if we set |Am| = |Aa;|, |Ag| \At\, n = 1/2 and 



= 1/2 



D 



\Ax\f 



2D\At\ 



(50) 



That is, in the context of the information transfer model 
we consider the square of an expected travelling distance 
(|Aa;|) of the tracer point particles (this is not the mean 
travelling distance, because Aa; = 0) but rather the mean 
square distance Aa;^. Furthermore, the diffusion coeffi- 
cient D is considered in the context of the information 
transfer model more generally as a reference constant of 
an IIT process (in this case an IIT process with an in- 
formation transfer index k — 1/2). According to the 
local IIT process condition [5] the random walk process 
must become a local IIT process if the absolute value 
of the source variable becomes locally sufficiently small 
(|Aq| = I At I 0). For an IIT process we can apply the 
ideal point particle equation (j22p which now becomes 
(setting \ Aq\ = \ At\, \Au\ = |Aa;| and k = 1/2) 



Ax At 

= 2- ^ N » 1 

\6x\ \5t\ 



(51) 



We see that the process variable selections, 
|Aa;| / |5x| or |At|/|(5t[, are equal or, respectively, 
proportional to the considered huge number N of 
randomly walking local tracer particles. This guarantees 
that the ideal transfer law condition [7] is also valid if 
|Ax| is in the range of the average jump length of the 
considered tracer particles. 

The experimental study of a random walk process im- 
plies the collective observation of a huge number of single 
tracer particles, so that one can define a local tracer par- 
ticle flux 



1 dN 



(52) 



where ON is the infinitesimal number of tracer particles 
which pass through the unit area Ao in x direction during 
the infinitesimal time interval dt. The local concentration 
C of the tracer particles is given by definition as 



C 



1 dN 
Ao dx 



(53) 
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We can now consider the local ratio between flux and 
concentration 



M 
c 



\6t\ 



(54) 



detector 



which yields a virtual detector {\d x\ / 1<5 il)(jgt(,(,joj. in the 
context of the information transfer model. Because the 
random walk process is locally an IIT process we can ap- 
ply the ideal detector equation lfT2|) so that equation (l54|) 
becomes 



\jx\ 

C 



\Sx\ 

W\ 



detector 



l\Ax\_ 
2lAi| 



(55) 



The tracer particles obey locally the ideal point particle 
condition [H (|Aa;|) — |Ax|, so that we can combine the 
ideal transfer law ([50)1 and the ideal detector equation 



DC=\j^\\Ax\ 



(56) 



We can now eliminate the explicit absolute value of the 
destination variable, |Aa;|, if we apply a partial differen- 
tiation of equation (j56p for a given local flux jx 



d{DC) 






dx 




ax 



\3x\ 

which yields Pick's first law (e.g. [6] p. 1) 



Jx 



-D- 



dx 



(57) 



(58) 



if we omit the absolute value bars. This discussion shows 
that Pick's familiar first law is, in the context of the infor- 
mation transfer model, the result of a local IIT process. 



VI. IDEAL GLOBAL POINT PARTICLE 
PROPAGATION PROCESSES 

We consider a source of propagating point particles 
which is located at r = and assume that we have a 
technical measurement device at the detector point in 
Fig. [3] which detects the propagating particles located 
originally in the source point and which delivers a mea- 
surement value with the dimension of a velocity 



technical detector — ( — 
t 



(59) 



detector 



This value measured by the detector is always a superpo- 
sition of the contribution which results from the relative 
movement of the detector relative to the particle source 
and the contribution which results from the considered 
particle propagation process 



Jt 



detector 



6 r 
It 



rel. mov. 



6 r 
Jt 



(60) 



detector point 
t > 0, r> 



source point 



FIG. 3: Illustration of an irreversible point particle propaga- 
tion process. The process variable r is the distance from the 
infinitesimally small source point at r = 0, and the process 
variable t is the time elapsed since the beginning of the prop- 
agation process. At time t > we will locally detect point 
particles in the detector point at r > 0. 



To apply the information transfer model we must con- 
sider a detector without any movement relative to the 
particle source {{S r /S t),.^, mov. = 0). 

The particle source is embedded in a (mathematically) 
infinite volume so that we can consider the distance r as 
the global information destination of the particle prop- 
agation process. The age t (in the sense of a duration 
or, quite generally, of a limited absolute value of time) 
is limited and can be considered as the global informa- 
tion source of the particle propagation process. Accord- 
ing to the global IIT process condition [5] the propaga- 
tion process of particles must become a global IIT pro- 
cess if the distance from the particle source is sufficiently 
large (r = Ugiobai — * oo). For an IIT process we can ap- 
ply the ideal detector equation (fT^ (setting |Au| = r 
and \Aq\ = t) 



detector 



= K- with r > 0, t> Q 
t 



(61) 



where the detector value {5 r / 5 1) ^^^^^^^^^ is always positive 
for propagating particles so that we can omit the absolute 
value bars. The only constant which is to be determined 
experimentally is the information transfer index k. If we 
have no appropriate experimental detector device or no 
information concerning r or t, the information transfer 
index n can also be derived from the ideal transfer law 
(PT|) of a floating information source (setting |Au| — r 
and \Aq\ = t) 



.1/k _ 



1 



— —Rut 

K 



(62) 



propag. 



where (r) is the expected distance of the propagating 
particles from the particle source at the age t and i?K is 
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the reference constant of an IIT process (a particle prop- 
agation process in this case). Notice that r in the ideal 
detector equation ((6T|) and (r) in the transfer law ((62|) 
represent usually two different distances from the parti- 
cle source. The detector position r is controlled by the 
user of the detector device, whereas the expected distance 
(r) is a floating distance value which floats according to 
the transfer law (p^ with the age t of the particle prop- 
agation process. An experimentalist can control (r), at 
best, indirectly by controlling the age t of the process 
(i.e., the duration of the experiment). 

We will now apply equations (pTjl and ([S^ to two 
generic processes: First, the propagation of tracer parti- 
cles during the diffusion in solids, and second, the prop- 
agation of photons in vacuum over large distances. 



A. The tracer particle diffusion process 

We consider the diffusion process of tracer particles in 
an infinite homogeneous isotropic medium without the 
action of any external forces. The tracer particle source is 
located in the centre of a spherical coordinate system (the 
source point in Fig. [3]). The ideal detector equation at 
the detector point in Fig. [3] now becomes (using equation 
(|6T|) and setting k = 1/2) 



C 



Sr 
Jt 



detector 



r 

2t 



(63) 



where is the radial tracer particle flux and C is the 
concentration of tracer particles. In subsection |V] it was 
shown that k = 1/2 for a tracer diffusion process. The 
validity of the ideal detector equation ([6T|) requires a suf- 
ficiently large distance of the detector from the particle 
source. 

One can directly prove that the ideal detector equa- 
tion (|63p is valid for infinitesimally small tracer particle 
sources. For that we consider the solution for the tracer 
particle concentration distribution for a point diffusion 
source in an infinite volume (Q p. 29) 



C 



M 



8(7rL>t)3/2 



exp 



ADt 



(64) 



where M is the total amount of diffusing tracer parti- 
cles and D is the diffusion coefficient. The radial tracer 
particle flux, jr, is given by (e.g. 8J) 



-D 



.dC_ 
dr 



(65) 



Using solution ([64)1 we get equation (|63ll . It is, however, 
well known from diffusion theory (e.g. [8]) that a realistic 
tracer diffusion source with a given radius i? > can be 
approximated as an infinitesimally small tracer diffusion 
source if we consider the diffusion process at a distance 
r sufficiently far away from the tracer diffusion source 
(r >> R). That is, equation is also valid for realis- 
tic tracer particle sources if we restrict our discussion to 



sufficiently large distances {r >> R) of the detector from 
the particle source. 

Let us now assume in a thought experiment that we 
have a small freely movable virtual tracer particle de- 
tector device which is able to measure separately jr (r, t) 
and C (r, t) at any distance r in the medium and which 
yields the measured ratio jr/C as output value. If we 
could move such a virtual detector device in the medium 
at low temperatures (a frozen tracer particle distribu- 
tion) the output result would be equal to the relative 
velocity between our tracer particle source and our vir- 
tual detector. Our virtual detector device at rest would 
measure jr/C = at very low temperatures because of 
jr — 0- At sufficiently high temperatures the tracer par- 
ticle flux jr becomes detectable and we can measure a 
detector output value, jr/C. According to the global 
IIT process condition [S] the tracer diffusion process must 
become a global IIT process if the distance r from the 
tracer particle source is sufficiently large. That is, our 
virtual detector device will then deliver an output value, 
jr/C, according to equation ([55)) . A special feature of 
the ideal detector equation ([55]) is that we can calculate 
the age t of the tracer diffusion process by the output 
value, jr/C, of our virtual detector device if we know the 
distance r of the virtual detector device from the tracer 
particle source. 



B. The photon propagation process in vacuum 

Let us now apply the obtained relations to a cosmo- 
logical propagation process. Redshift occurs when the 
electromagnetic radiation that is emitted from an object 
is shifted toward the red end (less energetic end or longer 
wavelength) of the electromagnetic spectrum. The red- 
shift parameter z > is used to describe the change in 
wavelength. It is defined as 



Ao ^ Ae 
Ae 



(66) 



where Ag is the emitted and Ao is the observed wave- 
length. Conversely, a decrease in wavelength is called 
blueshift (z < 0). 

Redshift measurements are an important method in 
astronomy to get information about the radial velocity, 
Vr, of a radiation source (star or galaxy) relative to the 
observer (e.g. P p. 100) 



, , Vr 1 dr 

z{Vr) = — - — 

c c at 



(67) 



where dr is the infinitesimal radial displacement of the 
radiation source during the infinitesimal time interval dt 
and c is the vacuum light velocity (we restrict this discus- 
sion to Vr << c). This velocity related redshift in equa- 
tion ()67|) reflects our experimental experience for photon 
emission sources which are not very far away from the 
observer (on the cosmological length scale) . 
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The photons detected during a redshift measurement 
have their origin in a star or a galaxy which can always 
be considered as the source point in Fig. [3] at sufficiently 
large distance r. The detector point in Fig. [3] is the loca- 
tion of our laboratory in three dimensional space where 
we measure the redshift at process age t > 0. At process 
age t = the radiation source started to emit the first 
photons. We now consider the term z x c as the output 
value of a redshift measurement device 



S r 
Jt 



(68) 



detector 



which yields a detector in the context of the proposed 
information transfer model. If we consider the case where 
the redshift measurement device has no radial velocity 
relative to the photon source equation (67) would yield 
the result z = z{vr) = 0. Applying, however, the ideal 
detector equation (I6ip for propagating particles we have 
in this case 



S v\ T 

z{r, t) c = [ — ] = K- if = 

" ^ / detector ^ 



(69) 



which yields a distance and time dependent red shift 
value z = z{r,t). According to the global IIT process 
condition O the process of propagating photons must be- 
come a global IIT process if the distance r from the pho- 
ton source is sufficiently large. That is, our detector (a 
redshift measurement device) will then deliver an output 
value according to equation (|69p . To determine the un- 
known information transfer index k we consider the ideal 
transfer law of the process. In subsection IVl we had con- 
cluded by means of the ideal transfer law that k = 1/2 for 
a random walk process (see equation (|50)) ). For a photon 
propagation process the ideal transfer law ([S^ becomes 



(r) = i?« 



t 



(70) 



because it is experimentally well confirmed that electro- 
magnetic radiation propagates with a constant velocity, 
where (r) is the expected distance of the photons from 
a photon source of age t. The reference constant -Rk=i 
(with the dimension of a velocity) is a medium dependent 
constant which is equal to c in vacuum. We can now con- 
clude that K ~ 1 for a photon propagation process so that 
the ideal detector equation (|69p finally becomes 



z{r^t) c ^ - if Vr — G 



(71) 



where r is the distance of the detector (a redshift mea- 
surement device) from the observed photon source and t 
is the age of the observed photon source (star or galaxy). 
That is, if we know the distance r from a star or galaxy 
and are not moving relative to the star or galaxy (u^ = 0) 
we can determine its age t from redshift measurements 
(if the distance r is sufficiently large). Or vice versa, if 
we know the age of a star or galaxy we can determine its 
distance if Vr = 0. 



1000 
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-500 
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FIG. 4: Hubble's published radial velocity-distance relation 
among extra-galactic nebulae (see for the meaning of the 
different points and lines). 



If we can only measure distances r and redshifts z from 
the observable galaxies, a reasonable assumption is that 
the observable galaxies have comparable ages t^. Plotting 
such measurements in a diagram enables us to estimate 
the average age to of the observed galaxies. Hubble [l3| 
published distance dependent redshift measurements in 
1929. Fig. m shows Hubble's published velocity of each 
galaxy in his sample relative to the Milky Way. The pro- 
portionality constant, i/o = (^o) i is called the Hubble 
constant. Using this Hubble constant equation ([7T|) be- 
comes 



c z{r) = Hq r if u,. = 

and tn = = constant 

Ho 



(72) 



The discussion in our paper shows that Hubble's con- 
stant, Hq, can correspond only to the average age of all 
observed galaxies because every single galaxy shows a 
distance dependent redshift according to equation (j7ip . 
where t is the age of the observed galaxy. Interestingly 
enough the proposed information transfer model yields 
results which are in agreement with cosmological obser- 
vations concerning the redshifts of stars and galaxies: 
Narlikar and Arp [ll| state "the Hubble constant is sim- 
ply determined by the age of the galaxies which comprise 
the relation" (see our equation ([7^ 1. 



VII. SUMMARY 

Inequality ([S]) is a well known fundamental law from in- 
formation theory Q which states that any technical pro- 
cess can transfer at best the whole information from an 
information source to an information destination. To dis- 
cuss the consequences of this law for natural processes we 
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propose an information transfer model where the trans- 
fer system explicitly includes any natural processes. To 
develop this model we first simplified Shannon's diagram 
of a general communication system Q , where the trans- 
fer system consists of transmitter, channel, receiver and 
noise source, by our diagram in Fig. [T] The basic idea is 
that we can then formally replace the technical process by 
any natural process which can be described sufficiently by 
two independent generic process variables q and u (con- 
dition [T|) . Because of the independence of the generic 
process variables we can now formally consider them as 
information source q and information destination u of the 
information transfer model (Fig. [T]) . 

Starting from this point of view it was necessary to 
find a definition of information which is directly related 
to process variables. For that we defined process variable 
selections, applied the definition of information which 
was originally given by Hartley [s'] and obtained finally a 
working expression for the information based on the two 
independent process variables (equation ([5])). In the next 
step we defined an IIT process (definition [1]) which en- 
ables ideal information transfer from the source variable 
to the destination variable (condition |4]) . 

Because it is forbidden that any process (technical or 
natural) transfers more information to the information 
destination than the information source sends we derived 
sufficient conditions for a global IIT process and a local 
IIT process. We demonstrated that one can derive the 
ideal gas law, the radioactive decay law, the formation 
law of vacancies in crystals and Bubble's law if one ap- 
plies the global IIT process condition [S] Applying the 
local IIT process condition ^ allowed us to derive Fick's 
first law. 

There is no doubt that one cannot violate a funda- 
mental law (inequality ([5])) which is valid for technical 
processes if one replaces the technical process by any 
natural process which fulfils condition [T] Our abstract 
definition of information concerning generic process vari- 
ables, which is based on Bartley's Q approach, yields, 
interestingly enough, several well known laws from the 
simple model presented here. 



APPENDIX A: JUSTIFICATION OF IIT 
PROCESS CONDITIONS 

1. Justification of condition [5] 

According to condition [3] we have \6q\ > 0. That is, if 
we consider the case where the absolute value of a global 
source variable is limited (f/giobai = jAqj < oo) we have a 
limited number of source variable selections 



Considering, furthermore, the case that the absolute 
value of the global destination variable is allowed to be- 
come unlimited (ugiobai = l^^^l ^ oo) we would expect an 
unlimited number of destination variable selections 



^global 



oo if Wglobal OO 

and \Su\ < oo (A3) 



if we take only into account that \du\ is limited 
{\Su\ < oo) according to condition [31 However, accord- 
ing to inequality (jA2p an unlimited is forbidden for 
the considered case (ggiobai < oo). To solve this confiict 
we must conclude that n„ can never become infinity but 
remains limited according to the relation 

= — < oo if Ugiobai oo and ^global < oo (A4) 

no matter how large Ugiobai becomes. That is, if equation 
4| is correct for a given large Ugiob^i it remains correct 



for Ugiobai > ""global- Equation (jA4p represents condition 



El 



2. Justification of condition [6l 



We now consider the case that there is a huge number 
of destination variable selections, Uu, given so that we 
have KUu » 1. Because of inequality (|lip we now have 



1 << KTijj < Tig with < K < oo 



(A5) 



We would expect that the number of source variable se- 
lections becomes small (« 1) 



\Sq\ 



1 if |Ag| \dq\ > 



(A6) 



if \A.q\ becomes sufficiently small. However, according 
to inequality (|A5[) — > 1 is now forbidden for the con- 
sidered case {kUu >> 1). To solve this confiict we must 
conclude that Uq can never become a small number (~ 1) 
but remains a huge number (>> 1) according to the re- 
lation 



Uq = KUu >> 1 if |Ag| 







(A7) 



no matter how small |Ag| becomes. That is, if equation 
(IA7p is correct for a given small |Ag|* it remains correct 
for \Aq\ < \Aq\* . Equation (|A7[) represents condition |6l 



n„ — < oo if ^global < oo and \Sq\ > (Al) 



\Sq\ 



Because of inequality (|lip we have now 



KHu < Uq < oo with < K < OO (A2) 
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